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Abstract 

The Peaceman-Rachford splitting method is very efficient for minimizing sum of two functions 
each depends on its variable, and the constraint is a linear equality. However, its convergence was 
not guaranteed without extra requirements. Very recently, He et al. (SIAM J. Optim. 24: 1011 
- 1040, 2014) proved the convergence of a strictly contractive Peaceman-Rachford splitting method 
by employing a suitable underdetermined relaxation factor. In this paper, we further extend the 
so-called strictly contractive Peaceman-Rachford splitting method by using two different relaxation 
factors, and to make the method more flexible, we introduce semi-proximal terms to the subproblems. 

We characterize the relation of these two factors, and show that one factor is always underdetermined 
while the other one is allowed to be larger than 1. Such a flexible conditions makes it possible to 
cover the Glowinski’s ADMM whith larger stepsize. We show that the proposed modified strictly 
contractive Peaceman-Rachford splitting method is convergent and also prove 0(l/t) convergence 
rate in ergodic and nonergodic sense, respectively. The numerical tests on an extensive collection of 
problems demonstrate the efficiency of the proposed method. 

Key words: Semi-proximal, strictly contractive, Peaceman-Rachford splitting method, convex mini¬ 
mization, convergence rate. 

1 Introduction 

We consider the convex minimization problem with linear constraints and a separable objective function 

min 9\ (a:) + 0 2 {y), s.t. Ax + By = b, x £ X , y £ y , (1.1) 

where 0\ : R ni -A R and 62 : R" 2 -A R are continuous closed convex (could be nonsmooth) functions; 
A £ R mxni and B £ R mx ™ 2 are gj ven matrices; b £ R m is a given vector; X and y are nonempty closed 
convex subsets of R" 1 and R” 2 , respectively. Throughout, the solution set of (1.1) is assumed to be 
nonempty; and X and y are assumed to be simple in the sense that it is easy to compute the projections 
under the Euclidean norm onto them (e.g., positive orthant, spheroidal or box areas). 
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Let Cp(x, y , A) be the augmented Lagrangian function for (1.1) that defined by 


£p(x, V, A) := 9i(x) + 0 2 (y) - (A, Ax + By - b) + ^|| Ax + By - 6 || 2 , 


( 1 . 2 ) 


in which A £ K m is the multiplier associated to the linear constraint and /? > 0 is a penalty parameter. 
Based on the classic Douglas-Rachford operator splitting method [4], the alternating direction method 
of multipliers was proposed by Gabay and Mercier [ 8 ], Glowinski and Marrocco [10] in the mid-1970s, 
which generates the iterative sequence via the following recursion: 

x k+1 = argmin£g(iE, y fe , A fc ), (1.3a) 

x$LX 

y k+1 = argmin Cy{x k+1 ,y, A fe ), (1.3b) 

y&y 

X k+1 = X k _ ^ Ax k+1 + Byk+1 _b)_ (1.3c) 

Based on another classic operator splitting method, i.e., Peaceman-Rachford operator splitting method 
[18], one can derive the following method for ( 1 . 1 ): 

c fc+1 = argmin Cp(x, y k , X k ), (1.4a) 




\ k +h = \k _ /3(Ax k+1 + By k+1 - 6 ), 

y k+1 = argmin Cp(x k+1 , y, X k+ ^), 
y&y 

. A fc+1 = A fc+ 5 - /3(Ax k+1 + By k+1 - b). 


(1.4b) 

(1.4c) 

(l-4d) 


While the global convergence of the alternating direction method of multipliers (1.3a)-(1.3c) can be 
established under very mild conditions [1], the convergence of the Peaceman-Rachford-based method 
(1.4a)-(1.4d) can not be guaranteed without further conditions [2]. Most recently, He et al. [12] propose 
a modification of (1.4a)-(1.4d) by introducing a parameter a to the update scheme of the dual variable 
A in (1.4b) and (1.4d), yielding the following procedure: 

c fc+1 =argmin£ / 3 (a;,/,A fe ), 


xex 


\ k +i =X k - a/3(Ax k+1 + By k+1 - b), 

y k+1 = argmin Cp(x k+1 ,y, X k+ ^), 
yey 

. A fc+1 = A fe+ 5 - aj3{Ax k+1 + By k+1 - b). 


(1.5a) 

(1.5b) 

(1.5c) 

(1.5d) 


Note that when a = 1, (1.5a)-(1.5d) is exactly the same as (1.4a)-(1.4d). They explained the nonconver¬ 
gence behavior of (1.4a)-(1.4d) from the contract perspective, i.e., the distance from the iterative point 
to the solution set is merely nonexpansive, but not contractive. The parameter a in (1.5a)-(1.5d) plays 
the essential role in forcing the strict contractiveness of the generated sequence. Under the condition that 
a £ (0,1), they proved the same sublinear convergence rate as that for ADMM [14]. Particularly, they 
showed that (1.5a)-(1.5d) achieves an approximate solution of (1.1) with the accuracy of 0(l/t) after t 
iterations 1 , both in the ergodic sense and the nonergodic sense. 

Note that the parameter a plays different roles in (1.4b) and (1.4d): the former only affects the update 
of the variable y in (1.3b) while the latter is for the update of the dual variable A. Hence, it is natural to 
choose different parameters in these two equalities. In this paper, we give such a scheme by introducing 
a new parameter 7 in (1.4d), i.e., the dual variable is updated by the following manner: 


A fc+i = A fc+| _ + By k+1 - b). (1.6) 

1 As the work [16,17] and many others, a worst-case 0(l/t) convergence rate means the accuracy to a solution under 

certain criteria is of the order 0(l/t) after t iterations of an iterative scheme; or equivalently, it requires at most 0(l/e) 
iterations to achieve an approximate solution with an accuracy of e. 
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For convenience, we first introduce the whole update scheme of the modified strictly contractive semi- 
proximal Peaceman-Rachford splitting method (sP-PRSM) as 


x k+1 = argmin Cp(x,y k ,\ k ) + h\x - x k \\ 2 s , (1.7a) 

x£X Z 

X k+ i =X k -a/3(Ax k+ 1 +By k -b), (1.7b) 

y k+1 = arg min Cp(x k+ \ y, X k+ i) + \ \\y - y k f T , (1.7c) 

v&y 2 

k X k+1 = \ k+ ? - j/3(Ax k+1 + By k+1 - b) (1.7d) 

where S and T are two positive semi-definite matrices. In applications, by choosing different matrices S 
and T customizing the problems’ structures, we can obtain different efficient methods. 

Our main contributions are twofold. 


1. Motivated by the nice analysis techniques in [12] and [19], we proved that the sequence generated 
by sP-PRSM is strictly contractive and thus convergent, under the requirement that 


a £ [ 0 , 1 ), 


( 1 . 8 ) 


Moreover, we proved that sP-PRSM is 0(1/t) sublinearly convergent both in the ergodic and 
nonergodic sense. Note that the nonergodic convergence rate requires that 7 £ (0,1]. We remark 
that the convergence of sP-PRSM (1.7) can unify that of several existing splitting methods. 

• Choosing a = 0 ,7 £ (0, (1 + Vb)/2) and T = 0, S = 0, sP-PRSM (1.7) reduces to the classical 
ADMM while the convergence coincides with that of ADMM [9] ; 

• Setting a = 0,7 £ (0, (1 + v / 5)/2) and T > 0, S 0, sP-PRSM (1.7) covers the semi-proximal 
ADMM considered in [3,7,14,19] and the corresponding convergence results; 

• Setting a = 7 £ (0,1), and T = 0, S = 0, sP-PRSM (1.7) reduces to the strictly contractive 
PRSM proposed in [12] and the convergence of the two methods is identical. 


2. We added a proximal term to each of the two main subproblems in updating the x and the y 
variables. In fact, for ADMM, Eckstein [5] and He et al. [11] have already considered to add 
proximal terms to the subproblems for different purpose. Recently, Fazel et al. [7] proposed to 
allow S and T to be positive semi-definite, in contrast to the positive definite requirements in the 
classical algorithms, which makes the algorithm more flexible. 


The rest of this paper is organized as follows. In Sect. 2, we give the optimality condition of (1.1) by 
using the variational inequality and also list some assertions which will be used in later analysis. In Sect. 
3, we first give the contraction analysis of sP-PRSM (1.7), and then establish the global convergence. 
We discuss the sublinear and linear convergence rate in Sects. 4 and 5, respectively. In Sect. 6 , we 
test a variety of problems to show the efficiency of the proposed sP-PRSM (1.7). Finally, we make some 
conclusions in Sect. 7. 

On May 25, after attending the first author’s thesis defense, Prof. He told us that they had also 
considered to use different stepsizes a and 7 in (1.7) with T = 0 and S = 0, and we got their manuscript 
on May 26 [13]. We found that they established the convergence when 

7 £ ( 0, 1 + 2 5 j , a€ (-1,1), a + 7 > 0, |a| < I+ 7 - 7 2 . (1.9) 
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Notice that (1.9) covers the case when a < 0. If we restrict a > 0 in (1.9), then the corresponding 
relation will be 

a€ [0,1), 7 € (o, 1 + V 2 5 ~ —)■ (1-10) 

By some simple calculations, it is easy to see that the domain defined by (1.10) is a bit smaller than that 
defined by (1.8). By some private communications with Prof. He, we learned that they also discovered 
the formula (1.8). However, to provide the intuitive understanding and the unified convergence analysis, 
they presented (1.9) in [13] which connects a and 1 — 7 + 7 2 directly. 


2 Preliminaries 

In this section, we give the optimality condition of (1.1) and some notations or relations which will be 
frequently used in our analysis. Let Cl = X x y x R m . Throughout this paper, we make the following 
assumption. 

Assumption 2.1. Let Cl* C Cl be the set whose elements are the optimal solutions of (1.1) and the 
associating dual solutions of (1.1). Throughout the paper, we assume that fl* is non-empty. 


2.1 Optimality condition of (1.1) 

Owing to the convexity of 9 1 (-) and $(•), there exist two positive semidefinite matrices Si and £2 such 
that for any x, x' £ 



(2.1) 

£ l" 2 


0>\\y~y'\\k, 

(2.2) 


where £ y £ d9 2 (y ), £' £ d6i(y'). 
Denote 


u = 


y 


, v 


A 


w = | y | , and F(w) = 
A 


/ 6 r - AL T A 
tv - B T A 
y Ax + By — b 


Due to the convexity of 9 1 (-) and 9 2 (-), it is easy to show that the operator F(-) is monotonic. Specifically, 
for any w, w' € Cl, we have 


(w — w ', F(w) — F(w')) = 




> llu-u'IlL 


(2.3) 


where £ = 


£1 0 


\ 0 £ 2 j 

w* £ D* if there holds that 


\ v v-y' )’\ ty-Cy // 

and the inequality is due to ( 2 . 1 ) and ( 2 . 2 ). As shown in [ 12 ], we say that 
{w-w*,F(w*))>0, WweCl. (2.4) 


From Theorem 2.3.5 in [ 6 ] or Theorem 2.1 in [14], we can see that Cl* is closed and convex, and it can be 
reformulated as 

Cl* = Q {w £ Cl: (w-w, F(w)) > 0}. (2.5) 
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Let S denote the feasible set of (1.1), namely, 

S = {(x,y) : Ax + By = b,x G X,y &y} 

and denote V = S x M m . Let us restrict w £ fl in (2.4) and (2.5) to be w £ V C fi. By some suitable 
modification of the proof of Theorem 2.1 in [14], it is easy to show that the optimal solution set f2* can 
be characterized as 

fl* = P]{weil: (w-w,F(w))> 0}. (2.6) 

We'D 

Similarly, we say that w* £ fl* if there holds that 

{w-w*,F(w*))> 0, \/w£V. (2.7) 

Note that the optimality condition (2.7) will be frequently used in the convergence analysis. Owing to 
(2.7), we call w to be an e solution of (1.1) if 

sup {w — w,F(w)) < e. (2.8) 

weD 


2.2 Some notations 

Given two real matrices C and D of the same dimension, we define (C, D) = tr {C T D), where tr(-) is the 
trace operator. We use || • || to denote the 2-norm of a vector. We denote || 2 :||q = z T Gz for z£l" and 
G £ K" xn . For a real symmetric matrix S, we mark S 0 (S >- 0) if S' is positive semidefinite (positive 
definite). To make the analysis more elegant, we use r k = Ax k + By k — b for short. Similarly, for any 
w £ 2?, we denote r(w) = Ax + By — b. Obviously, there holds that r(w) = 0 for any w £ V. For ease of 
the analysis, we define the following matrices as 


H = 


1 


(a + 7 — a"f)/3B T B —aB J 


a + 7 


—aB 




and 


M = 


In, 0 


a/3B (a + j)pi m J 
We can easily verify that H is positive semidefinite if 0 < a < 1 and 7 > 0, and 

/ ( l-a)/3B J B 0 \ 


M HM = 


0 


(a + 7 )/3I„ 


Denote P = 


S 0 
0 T 


and define 


and 


G := 


P 0 
0 0 


G := 


E 0 
0 0 


0 0 
0 H 


G = 


( s 


0 


0 


0 T+ a+ ^-^ l3B T B 


V 0 -^=B 


a+7 

_I_ T 


f s + Ei 
0 

V 0 


t + h 2 + ^^pb j b ~^b t 


a+7 

- Z-B 

a+7 


1 T 
(a+ 7 )/3 ±m 


(2.9) 


( 2 . 10 ) 


( 2 . 11 ) 


( 2 . 12 ) 


(2.13) 
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where the dimension of zero matrix can be easily identified from the context. For any w,w' G S2, there 
hold that 

\\w-w / \\% = \\u- u Vp+\\v- v '\\ 2 h (2.14) 

and 

\\w-w'f d =\\u-u'\\l + \\w-w'\\ 2 G . (2.15) 

Finally, we present some relations of the iterates. They will play a crucial role in the convergence 
analysis. With the update scheme (1.7b) and (1.7d), it is easy to have 


X k = A fc+i + ( a + 7 )^ r fc+i + a /3B(y k - y k+1 ). 
Recalling the definition of v, with (2.16) and (2.10), we conclude that 


v k _ y k +1 = 


qjk _ \ / _ „.fc+l 

y y = M y y 


X k _ X k +1 


r k-\-l 


With (2.11) and (2.17), we thus have 

\\v k - v k+1 \\ 2 H = (1 ^ a)/3\\B(y k - y k+1 )|| 2 + (a + 7 )/3||r fc+ 1 1| 2 . 


(2.16) 


(2.17) 


(2.18) 


3 Convergence of sP-PRSM 

In this section, we first show that the sequence {u>fc} generated by sP-PRSM (1.7) is strictly contractive 
and thus can establish the convergence of the method. With the help of the contraction property, we 
further discuss the convergence rate in the ergodic and nonergodic sense. 

3.1 Contraction analysis 

We first give the contraction result as follows, and then complete its proof at the end of this section. 
Note that the following result will play a key role in proving the convergence of (1.7). 

Theorem 3.1. Let the sequence {u> fc } be generated by sP-PRSM (1.7). If we choose a and 7 according 
to (1.8), then for any w* £ W*, there holds that 

— ^llg + da,7 Ill'll 2 + Voi^Wyb — y k X ||t) 

- (||u ; fe+1 - w*\\ 2 e + p a ^\\r k+1 \\ 2 + r] a}1 \\y k+1 - y k \\^j > r a ^\\w k - w k+1 1||, (3.1) 

where r aj7 = min(i,T a>7 ), and > 0 and 0 < t q , 7 < 1 are some constants which only depend on 

a and 7 . 

To prove Theorem 3.1, several lemmas should be established first. The following Lemma 3.1 is mainly 
based on the optimality conditions of (1.7a) and (1.7b). 

Lemma 3.1. Let the sequence {w k } be generated by sP-PRSM (1.7). If a > 0 and 7 > 0, then for any 
w€V, there holds that 

(w — w k+1 ) T G(w k+1 — w k ) > (1 — a — j)/3 ||r fc+1 1 |~ + (1 — a)/3 (r k+1 , B(y k — y k+1 )) 

+ (w k+1 -w,F(w k+1 )) . (3.2) 
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Proof. Consider any £ k+1 £ d6i{x k+1 ) and f k+1 £ d02(y k+1 ). We then obtain respectively from the 
optimality conditions for (1.7a) and (1.7c) that 

(x - x k+1 ,S{x k+1 - x k ) + £ k+1 - A T X k + (3A T r k+1 + /3A J B(y k - y k+1 )) >0, Vz £ X 

and 

(y ~ y k+ \ T{y k+1 - y k ) + ^ +1 - B T A fc+ ^ + pB J r k+l ) >0, Vy £ 3>. 

Substituting (2.16) and A fc+ s = A fe+1 +7 /3r k+1 which follows from (1.7d) into the above two inequalities, 
respectively, we obtain the new formulations of the optimality conditions for (1.7a) and (1.7c) as 

(x - x k+1 ,S(x k+1 - x k ) + - ^4 T A fe+1 + (1 - a - 7 )/3A J r k+1 + (1 - a)/3A J B(y k - y k+1 )) > 0, 

Vx £ X. (3.3) 


and 

(y-y k+ \T(y k+1 -y k )+f k+1 -B J \ k+1 +(l- 1 )f3B J r k+1 )>0, Vy £ y. (3.4) 

Rewriting (2.16) to be 

r k+1 - ^B(y fc+1 - y k ) + . * , J A fc+1 - A fe ) = 0. (3.5) 

a+ 7 (a + 7)p 

Combing (3.3), (3.4) and (3.5) in a suitable way, and recalling the definition of F(w ), we see that for any 
w £ fl there holds that 


w — w 


fc+i 


( S{x k+1 - x k ) 
T(y k+1 -y k ) | + 
0 


/ 0 

a/3B J r k+1 + (1 — a)/3B T B(y k+1 — y k ) 
V ^B(y k +' - y k ) + (^(A fc+1 - \ k ) 


> ( w k+1 - w, 


/ .4 T 

B J I [(1 - a - j)f3r k+1 + (1 - a)f3B(y k - y fc+1 )] ) + <«; fc+1 - w, F(w k+1 )) . (3.6) 

V 0 


With the relation (3.5) and the definition (2.9) of H, by some easy calculations, we obtain that 


ot/3B T r k+1 + (1 - a)(3B J B(y k+1 - y k ) \ _ fc+1 _ . 
-^B(y k +' y k ) + ^(A^ - X k ) ) ~ [V 


(3.7) 


Considering the definition of r k+1 and r(w), we have ( A , B , 0)(w fc+1 — w) = r k+1 — r(w). This, together 
with (3.6) and (3.7) indicates that 


(w - w k+1 ) J G(w k+1 - w k ) > (r fe+1 - r(w), (1 - a - y)/3r k+1 + (1 - a)(3B(y k - y k+1 )) 

+ (w k+1 - w,F(w k+1 )) . 


(3.8) 


Notice that r(w) = 0 for any w £ V 1 we can immediately obtain (3.2) from (3.8) since The proof is 
completed. □ 

Lemma 3.2. Let the sequence {w k } be generated by sP-PRSM (1.7). If a > 0 and 7 > 0, then for any 
w £ T>, there holds that 

\\w k -w\\ 2 g - \\w k+1 -w\\% > \\u k - u k+1 \\ 2 P + (1 -a)!3\\B{y k -y fc+1 )|| 2 + (2 - a - 7)/3 ||r fe+1 ||~ 

+ 2(1 - a)/3 (r k+1 , B(y k - y k+ 1 )> + 2 (w k+1 - w, F{w k+ 1 )> . (3.9) 
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Proof. Using the identity 


\\a\\ 2 G -\\b\\ 2 G = \\a-bf G + 2b T G(a-b), 


with a = w — w k and b = w — w k+1 , we have 


\w k - w\\ 2 g - \\w k+1 - w\ \ 2 g = | k fc - w k+1 " 2 


\g + 2(w - w k+1 ) T G{w k+1 - w k ). 
Plugging (3.2) into (3.10) with (2.14) and (2.18), we have (3.9). The proof is completed. 


(3.10) 

□ 


Denote K = 
be expressed as 


(1 - a)/3B J B (1 - a)/3B T 
(1 - a)/3B (2 - a - 7 )/3I„ 


. With (2.10) and (2.17) , the inequality (3.9) can 


\\ wk ~ w \\g - \\ wk+1 - w \\g 

> ||u fe - u fe+1 ||p + ( v k - v k+l ) T M~ J K M~ l {v k - v k+1 ) + 2 ( w k+1 - w,F(w k+1 )) . (3.11) 


Our task now is to estimate the first term in the righthand side of the inequality in (3.11). 

Consider the case where 0 < a < 1 and 0 < 7 < 1. It is easy to see that K >~ 0. Moreover, by some 
tedious calculations, we conclude that 

K>c■ M J HM , 

where c = 1 ^ 6 (0,1). Thus we obtain from (3.11) and (2.14) that 

||u> fe - w\\ 2 G - ||w fe+1 - w\\ 2 g > c\\w k - w k+1 \\ 2 G + 2 ( w k+1 - w, F{w k+1 )) , (3.12) 

which is sufficient to establish that the sequence {||u> fe — w*||q} is strictly contractive by choosing w = w* 
in (3.12) and further to show the convergence of sP-PRSM (1.7). 

Consider the case when 0 < a < 1 but 7 > 1. The matrix K is not positive semidefinite any more. 
Instead of establishing the strictly contractive property of the sequence {||ui fe — w*||g}, we consider to 
prove the property of another sequence related to the sequence {||ie fe — wj*||q}. To do so, we need to give 
a more careful estimation of the intersection term (r k+1 , B(y k — y k+1 )). 

Lemma 3.3. Let the sequence {u> fc } be generated by sP-PRSM (1.7). If a > 0 and 7 > 0, then for any 
w £ Q, there holds that 

(r k+ \B(y k - y k+1 )) > ^ (r k ,B(y k - /+ 1 )) - -£-|| B(y k - y k+1 )|| 2 

+ \ ( ||yfc+1 - yk Hr - II y k - y k ~r T ) • (3.13) 

2(1 + a) p 

Proof. Note that the optimality condition (3.4) also holds with k := k — 1, then we have 

(y-y k ,T(y k -y k - 1 ) + Z k -B T \ k + (l-i)pB T r k )> 0, My £ y, (3.14) 

where f k £ 862(y k ). Choosing y to be y k and y k+1 in (3.4) and (3.14) and then rearranging the obtained 
inequalities, respectively, we have that 

(B(y k - y k+1 ),-X k+1 + (1 - 7 )/3 r k+1 ) > \\y k+1 - y k \\ 2 T + < y k+1 - y k ,£ k+1 ) (3.15) 


and 


(B(y k - y fe+1 ), A fe - (1 - 7 )/3r fe ) > - < y k+1 


y k ,T(y k — y k ~ x )) — (y k+1 


y^O- 


(3.16) 







Summing (3.15) and (3.16) over the both sides and then using the relation (2.16), we obtain that 


(1 + a)/3 < B(y k - /+ 1 ), r k+1 ) + ap\\B(y k - / +1 )|| 2 - (1 - 7 )/3 (B(y k - y k+1 ), r k ) 

> II/ +1 - /III + (y k+1 - v k > / +1 - /> - <: y k+1 - /, n/ - /- 1 )} 

> ^(ll/ +1 - /III - 11/ - y fc_1 ||r). (3.17) 

where the second inequality is due to (2.2) and the Cauchy-Schwarz inequality. It is trivial to obtain 
(3.13) from (3.17). The proof is completed. □ 


With the help of Lemma 3.3, we are ready to have the following theorem. 

Theorem 3.2. Let the sequence {w k } be generated by sP-PRSM (1.7). If we choose a and 7 according 
to ( 1 . 8 ) , then for any w there holds that 


(lk fc - wfa + p a ,4r k \\ 2 + y a J\y k - y k ~ x III) - (lk fc+1 - /l 2 G + P^b k+1 \\ 2 + Pa, 7 ll v k+1 
> r a , 7 \\w k - w k+1 \\l + 2 (w k+1 - w) J F(w k+1 ), 


/III) 

(3.18) 


where p a , 7,^,7 > 0 and 0 < r Qi7 < 1 are some constants which only depend on a and 7 . 


Proof. We consider three cases. 

I) . 0 < 7 < 1. With p Qj7 = r/a^ = 0 and r a>7 = 1 '^ 1 7> , the inequality (3.18) coincides 

with (3.12) which has been proved. 

Before we proceed, by combining (3.13) and (3.9), we derive the useful inequality as 

(lk fc - HIg + y^H/ + 1 - /III) - (lk fe+1 - w Wg + ^11/ - /■ 1 lll) 

> ||u fe - u k + l f P + _ y k + 1)||2 + (2 _ a _ 7)/? ||r fe+1 || 2 

1 + a 11 11 

+ 2(1 - 7)^/3 (r fc , B(y k - /+ 1 )) + 2 (w k+1 - w, F(w k+1 )) . (3.19) 

1 + a 

II) . 7 = 1 . It is not difficult to see from (3.19), (2.14) and(2.18) that (3.18) holds with p an = 0 and 
7a ,7 = 7a ,7 = i+fp 

III) . 1 < 7 < 1 a +y/P +< P + 4 ! 1 L_L % L e t us choose any 5 6 — 3 - 7 ^ ■ Note that this interval 

is well-defined due to the range of 7. With the Cauchy-Schwarz inequality, we know that 

- 2 (r fc , B(y k - /+ 1 )) > -6 ■ ||r fc || 2 - 1 • \\B(y k - /+ 1 )|| 2 . 

Plugging the above inequality into (3.19), we obtain that 


lk fe - HI g + TT ^ llyk+1 ~ yk ^ T ) ~ ( l|u;fe+1 “ HI 2 


1 - a, 


G T TTTtII/ - / III 

1 + a 


> \\u k -u k+l \\l 


( l -/ 2 

1 + a 


1 - 


7 — 1 1 

1 — a 


7 ) P\\B(y k — / +1 )||2 + (7 — 1)1—— 

0 J 1 + a 


1 — a /1 + a 1 + a 


1 + a \ 7 — 1 1 — a 


-6 


n k +1 


-< 5 ( 7 - 1 ) 


1 — a 
1 + a 


| r k || 2 - ||r fc+1 || 2 ) + 2 (w k+1 - w,F(w k+1 )) , 


which with (2.14) and (2.18) implies that (3.18) holds with p Qi7 = S(j — 1)tt^, Pa ,7 = tt^ and 


1 — a 

t„ ~ = -- mm 

’ 7 1 + a 




7 — 1 /1 + a 
a + 7 \7 — 1 



The proof is completed. 


(3.20) 

□ 
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Finally, we end this subsection by finishing the proof of Theorem 3.1. 

Proof of Theorem 3.1. Consider any w* £ ft*, recalling the monotonicity (2.3) of F(w ), we thus 
obtain from the optimality (2.4) of w* that 

(w k+1 -w*,F(w k+1 )) > (w k+1 -w*,F(w*)) + \\u k+1 > ||u fc+1 - m*|||. 


With the above statement, choosing w = w* 

(II w k - w*\\ 2 a + Pa, 7 \\^ k II 2 + ria G \\y k ~ y k ~ 
>T aG \\w k -w k + 1 f G + 2\\u k+1 -u*\\l, 


in (3.18), we can easily obtain (3.1). 

1 IIt) - (lk fe+1 - w*IIg+ p«, 7 ||r fc+1 || 


2 


+ Pa, 7 ||y 


fc +1 


y k III) 

(3.21) 


Adding the term \\u k — u*||| — ||w fc+1 — w*||i; on both sides of (3.21), we have 

(11?/' — U*||! + \\w k — W*\\ G + Pa, 7 |k fc ||" + Pa, 7 ||p fc - y k 1 ||t) 

- (||?/ +1 - «1| + II^ 1 - w*\\ G + Pa, 7 lk fe+1 ll 2 + y a J\y k+1 - y k IIt) 

> Ta G \\w k — w/ +1 |Ig + ll ufc+1 — u *lll + ll ufe — u *IIe 

> r aG \\w k - w k + l \\ 2 G + l -\\u k - u fc+1 |||, (3.22) 


which with (2.13) and (2.15) leads to (3.1). The proof is completed. 


3.2 Global convergence 

Theorem 3.3. Let the sequence {??/} be generated by sP-PRSM (1.7). If we choose a and 7 according 
to ( 1 . 8 ) , then {??/} converges to an optimal solution of ( 1 . 1 ). 


Proof. We divide the proof into three steps. 

(i) To show that the sequences {«/} is bounded. It is trivial to know from (3.1) that lim^oo \\w k — 
w k+l \\ 2 ~ = 0, which with (2.14), (2.15) and (2.18) indicates that 

Gx 

lim \\x k - a/ +1 || s+Sl = lim ||y fc - y k+1 \\ T+ ^ = Km ||r fe || = lim \\B(y k - y fc+1 )|| = 0. (3.23) 

AC—KX) K—tOO AC—KX) K—^OO 


By some simple calculations, there holds that there exists a constant cq = 
such that 


G h c 0 


f S + £1 
0 

V 0 


0 

T + S 2 + /3B J B 
0 



2 (a+ 7 )— ary—ay/ 7 2 +4(a:+7) 
2(q:+7) 


e (0,1] 


(3.24) 


Besides, it is straightforward to see from (3.1) that \\w k — w*\\% is bounded. This together with (3.24) 

Gr 

shows that 


|a/-z*|| s+Sl , ||?/ fe - ?/1 t +S 2 + pst b , IIA fc — A*| 


are all bounded. Obviously, we claim that {A fc } is bounded. Moreover, we see that {y k } is bounded as 
T + £ 2 + PB T B >- 0. Notice that || Ax k — Ar*|| = ||r fc + B(y k — p*)|| < ||r fc || + \\B(y k — p*)||, with (3.23), 
this implies that \\x k — x*\\^a t a is bounded. Recalling that S + Si + /3A T A >~ 0 and \\x k — t*||.S+Ei is 
bounded, it is safe to say that {x k } is also bounded. 

(ii) To show that any cluster point of the sequence {??/} is an optimal solution of (1.1). Let {w ki } 
be a subsequence of the sequence {«/} and lim w ki = w°°. Since the graphs of ddi(-) and 862 (•) are 

ki—too 

both closed, taking the limit with respect hi —> 00 on both sides of (3.2) and using (3.23), we have that 


(w - w°°) t F{w°° ) >0, Vw 6 V, 
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which means that .°° is an optimal solution of ( 1 . 1 ). 

(iii) To show that the sequence {w k } has only one cluster point. We first replace x* with x°° in the 
analysis of Steps (i) and (ii). It follows from w ki —> w°° that \\w ki — .°°||| + p a ,j\\r ki || 2 + Va,-y\\y ki — 
/ i_ 1 || j, —> 0. Owing to the monotonicity of the sequence ||. fc — .°°||| + p a , 7 l|r fe || 2 + Va,'y\\y k ~ / _1 |||, 
we can see that 

lim ||. fc - .-HI + p a , 7 ||r fe || 2 + ^11 y k - /"H = 0 . 

k—>oo ^ 

This together with (3.24) shows that 

lim ||rc fc - x-Hs+Ei = lim ||/ - 2/ 00 ||t+s 2 +/3stb = lim ||A fc - A°°|| = 0. (3.25) 

k—yoo k—yoo k—too 

Using again the inequality ||4:r fe —Ar°°|| = \\r k + B(y k —y°°)\\ < ||r fc || + ||.B(/ — p°°)|| and lim^oo ||rfc|| = 
0 , we have 

lim ||A(x fc -a; 00 )|| =0. (3.26) 

k—yoo 

Combing (3.25) and (3.26), and using that S + Ei + (3A J A y 0 and T + E 2 + f3B T B y 0, we immediately 
have 

lim w k = .°°. 

k—t 00 

The proof is completed. □ 


4 Sublinear convergence of sP-PRSM 

The rate of convergence of an algorithm can help us have a deeper understanding of the algorithm. 
Thus, in this section, we establish the sublinear rate of convergence of sP-PRSM, in ergodic sense and 
nonergodic sense, respectively. 


4.1 Convergence rate in the ergodic sense 

We now give the sublinear rate of convergence of sP-PRSM in the ergodic sense, which is very easy due 
to the key inequality (3.18). 


Theorem 4.1. Let the sequence {. fe } be generated by sP-PRSM (1.7). For any integer t > 0, define 

1 t 


™t = -J2 w 

k= 1 


,k +1 


Then for any w £ T>, we have that 


sup (wt-W,F(w)) < — (II. 1 - . 0 ||| + Pa, 7 lk 1 || 2 + T ?a, 7 ||2/ 1 -J/°|||) • 


(4.1) 


Proof. It follows from (3.18) and (. fc+1 — w) T F(w k+1 ) > (w k+1 — w) T F(w ) following from (2.3) that 
(w k+1 - .,F(.)) < i (||. fc - .||| + p a , 7 ||r fc || 2 + y a , 7 \\y k - / _1 III) 

- i (||. fc+1 - .III + Pa, 7 ||r fe+1 II 2 + ^, 7 ||y fe+1 - /III) 

For all k = 1,.. .,t, summing the above inequality from both sides and noting the notation of wt, we 
derive that 

t (w t — ., F(w)) < i (II. 1 -.°|||+Pa, 7 ||r 1 || 2 + 77 a>7 ||?/ 1 -/|||) . (4.2) 

Since (4.2) holds for any w £ D, we can easily have (4.1). The proof is completed. □ 


Remark: Consider the case when the set V is compact, (4.1) implies the 0{l/t) ergodic convergence of 
sP-PRSM (1.7). 
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4.2 Convergence rate in the nonergodic sense 

Lemma 4.1. Let the sequence {w fc } be generated by sP-PRSM (1.7). If 0 < a < 1 and 7 > 0, then 
w k+1 £ Ll*, namely, w k+1 is one optimal solution of (1-1), if 

\\w k -w k+1 \\ G = 0. 

Proof. Following from (2.14) and (2.18), the statement \\w k — u> fc+1 ||G = 0 indicates that r k+1 = 0. 
Noting that G is always positive semidefinite, it is trivial to have G(w k — w k+1 ) = 0. Plugging the above 
two assertions into (3.2), we see that (w — w k+1 , F(w k+1 )) > 0 holds for any w £ V. Following the 
optimality condition (2.7), we immediately see that w k+1 £ fl*. The proof is completed. □ 

Based on this lemma, we can use ||w fc — w; /c+1 ||g to measure the accuracy of w k+1 . Similarly to the 
definition ( 2 . 8 ) of e-solution of ( 1 . 1 ), we also call w k+1 as an e-solution of ( 1 . 1 ) when ||w; fc — u> fe+1 ||G < £■ 
Together with (2.14) and (2.18), this also provide a practical stopping condition for sP-PRSM (1.7) as 

max{||a ; fe+1 - * fc || s , \\y k+1 y k \\ T , || B(y k+1 - y k ) ||, ||r fc+1 ||} < tol, (4.3) 

where tol is some tolerance. Note that in (4.3), we may pay some more price to compute ||:r fc+1 — 
z fc ||s, || y k+1 — y k \\r- Thus, when the spectral norm of S' or T is bounded, we can also simply stop the 
method when 

max{||B(y fc+1 - /)||, ||r fe+1 ||} < tol. (4.4) 


Lemma 4.2. Let the sequence {w fe } be generated by sP-PRSM (1.7). If we choose a and 7 according to 
( 1 . 8 ), then there holds that 

\\w k+1 - w k \\l II w k+2 - w k+1 \\l > 1 - 2 ll (^ fc+1 - w k ) - (w k+2 - w k+1 )\\l. (4.5) 

a + 7 

Proof. Note that (3.8) also holds with k := k + 1, then we have 

(w - w k+2 ) T G{w k+2 - w k+1 ) > (r k+2 - r(w), (1 - a - 7 )/? r k+2 + (1 - a)j3B{y k+1 - y k+2 )) 

+ (w k+2 - w,F(w k+2 )) . (4.6) 

Choosing w to be w k+2 and w k+1 , respectively, in (3.8) and (4.6) leads to 

{w k+2 - w k+1 ) T G(w k+1 - w k ) > (r k+1 - r k+2 , (1 - a - 7 )^r fc+1 + (1 - a)/3B(y k - y k+1 )) 

+ (w k+1 - w k+2 , F(w k+1 )) . (4.7) 

and 

(■ w k+1 - w k+2 ) T G(w k+2 - w k+1 ) > (r k+2 - r k+1 , (1 - a - j)/3r k+2 + (1 - a)/3B(y k+1 - y k+2 )) 

+ ( w k+2 - w k+1 , F(w k+2 )) . (4.8) 

Adding (4.7) and (4.8) and noting (w k+2 - w k+1 , F(w k+2 ) - F(w k+1 )) > 0, following from (2.3), we 
obtain that 

{w k+2 - w k+1 ) J G [(w fc+1 - w k ) - ( w k+2 - w k+1 )] 

> (1 - a - 7)0|| r k+1 - r k+2 \\ 2 + (1 - a)/3 (B [( y k - y k+1 ) - ( y k+2 - y k+1 )\ , r k+1 - r k+2 ) . (4.9) 
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Following the deriving process of (2.18), we have that 


||(w fc+1 - w k ) - ( w k+2 - w k+1 )\\l 

= (1 - a)P \\B[{y k+1 - y k ) - (y k+2 - y k+1 )}f + (a + 7 )/3|| r k+1 - r k + 2 1 | 2 . (4.10) 


Thus we conclude that 

\\w k+1 - W k \\ 2 H -\\w k+2 -u, k+1 \\ 2 H 

= 2 (w k+2 - w k+1 ) T G |> fe+1 - w k ) - ( w k+2 - w k+1 )] + ||(u; fc+1 - w k ) - ( w k+2 - u> k+1 )\\ 2 H 

> (2 - a - 7)/3| \r k+1 - r k+2 1 | 2 + 2(1 - a)/3 (B [{y k - y k+1 ) - ( y k+2 - y k+1 )\ , r k+1 - r k+2 ) 

+ (1 ^ a)/3 \\B[{y k+1 - y k ) ( y k+2 - y k+1 )} f 

> (1 - l)P\\r k+l - r k+2 \\ 2 > ^^||(u ; fc+1 - w k ) - ( w k+2 - w k+1 )f G , 

a + 7 

where the first inequality is due to (4.9) and (4.10), the second inequality is trivial and the last inequality 
owes to (4.10). The proof is completed. □ 


Theorem 4.2. Let the sequence {u fe } be generated by sP-PRSM (1.7). If'y £ (0,1], then there holds that 


||in t+1 — u7|| Q < — • - ■ (||u7 - W°||g + Pa, 7 ||r 1 || 2 + Pa^Wy 1 — 2/ 0 II t) ■ 

‘CX., 7 ty 

Proof. Summing (3.21) over k = l,...,t leads to 


(4.11) 


T a,7' W wk+1 ~~ wk \\ 2 G < ll^ 1 ~ w *\\G + Pa,7l|?" 1 || 2 + Va^Wy 1 ~ V°\\T- 


(4.12) 


fc=l 

Observe that 7 £ (0,1], it follows from (4.5) that ||u ; fe+1 — w k \\ G > ||w fc+2 — w k+1 \\ G , which together with 
(4.12) can easily suggest (4.11). The proof is completed. □ 


5 Linear convergence of sP-PRSM (1.7) 

For simplicity, we only consider the case when T = 0 and S = 0. In this case, we always have that 

and \\w-w , \\ g = \\v-v'\\ 2 h . ( 5 . 1 ) 

Assumption 5.1. The set y = R” 2 . 

Assumption 5.2. The gradient V0 2 (-) is Lipschtiz continuous on M" 2 with L > 0. Thus for any 
2/i j 2/2 £ R n2 there holds that 

||Vd 2 (2/i) — V/?2(2/2)|| < L\\yi — 2/2 1 | ( 5 . 2 ) 

and 

(2/i - 2/2, V0 2 (yi) - V/M2/2)) > i||V 0 2 (i/i) - Vd 2 (y 2 )\\ 2 . ( 5 . 3 ) 

Assumption 5.3. The function 0 2 (-) is strongly convex on R" 2 with constant p > 0. Thus for any 
2/1,2/2 £ R” 2 there holds that 

(?/i ^ 2/2, V6» 2 (2/!) - yd 2 (y 2 )) > p\\yi - y 2 \\ 2 ■ ( 5 . 4 ) 


G = 
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Since y = R” 2 , it follows from (3.4) that 

V 6 > 2 (j/ fe+1 ) = B J X k+1 - (1 - 7 )/3 B J r k+1 . 
Similarly, we obtain from (2.7) that 

VW) = b t x*. 


(5.5) 


(5.6) 


Theorem 5.1. Let the sequence {u> fc } be generated by sP-PRSM (1.7). Under Assumptions 5.1, 5.3, 
5.2, there must exists some constant c > 0 such that 


\\v^ v *r H + Pa ^r>{i+c) 


fc+l * 112 


\\V — V 


||^ + Pa >7 ||r fe+1 || 2 ). 


Proof. Letting v = v* in (3.18), with (5.1), we can obtain 


(ll«‘ - v *\\h + P*,Jr k \\ 2 ) (lk fe+1 - v*\\h + P«, 7 lk fc+1 || 2 ) 
> r a ,Jv k - v k+1 \\ 2 H + 2 (w k+1 - w*,F(w k+1 )) . 


(5.7) 


Next we will estimate the two terms of the righthand side of (5.7), respectively. Firstly, with (2.18), we 
see that 

\\v k -v k+1 \\ 2 H >(a + 1 )P\\r k+1 \\ 2 - (5-8) 

It follows from (2.3), (2.4) and the convexity of 0i(-) that 


(w k+1 - w\ F{w k+1 )) > < y k+1 - y*,\7d 2 (y k+1 ) - V W)> 


(5.9) 


For any 0 < t < 1, consider the convex combination of (5.4) and (5.3), we know from (5.9), (5.5) and 
(5.6) that there holds that 

L 

By applying the inequality 


(w k+1 - w*,F(w k+1 )) > M (1 - t)\\y k+l - y *|| 2 + Amin(ggT) f . ||(A fc+1 - A*) - (1 - 7 )/3r fe+1 || 2 . (5.10) 


a + &|| 2 > 1 -- |H | 2 + (l-p)||&|| 2 , p> 1 


with a = X k+1 - A*, b = -(1 - 7 )r fc+1 and r > 1, we have 


ll(A fc+1 - A*) - (1 - 7 )r fc+1 || 2 >(!--) ||A fc+1 - A *|| 2 + (1 - p )| 7 - l|/3 • ||r fc+1 || 2 . 


(5.11) 


Plugging (5.11) into (5.10) , we further have 

<■ w k+1 - w*,F(w k+1 )) > Cl (m\ 2 ■ II y k+1 - y*\\ 2 + ^||A fe+1 - A*|| 2 ) + c 2 /3||r fe+ 1 1| 2 , (5.12) 

where ci = min (1 — t), (l — p _1 ) Amln ^ B ^ /3tJ , c 2 = (1 — p )| 7 — 1| Amin ^ B With the definition 

(2.9) of H , we can easily see that there exits a positive constant c Qj7 = 2 ^ a+7 ^ “ 7 such 


that 


(5.13) 


P\\B\\ 2 ■ II y k+1 - y*t + ^l|A fc+1 - A*|| 2 > c a , 7 • ||u fc+1 - v*f H . 

Combing (5.8), (5.12) and (5.13), we see that 

RHS of (5.7) > 2(ci • ca, 7 ) \\v k+1 - v*\\% + (2c 2 + a + 7 )/3||r fc+1 || 2 

> C (||u fc+1 — U*|||f + Pa, 7 |k fe+1 || 2 ) J (5.14) 

where c = min ^2cic ai7 , 2c2 p + a + 7 f^J . The proof is completed. □ 
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Remark. Let us consider as a special case, namely, 0 < a < 1,7 = 1. Here, p an = 0 and (5.11) reduces 
to ||(A fc+1 - A*) - (1 - 7 )r fe+1 || 2 = ||A fc+1 - A*|| 2 . Hence c = 2 cic 3 . Letting t = } , we 

have that 

= p(3\ min {BB T ) 1 

Cl pL + /3 2 ||i?|| 2 A min (i?H T ) - 2 K By fK^ 


where kb = \J X(BB T )/X m i n (BB T ), kq 2 = L/p and the equality attains when j3 
largest c is 

Cot, 7 

Cm ax — • 

2 

which reduces to (3.16) in [3] when a = 0. 


[iL 


. The 


(5.15) 


6 Numerical results 

In this section, we demonstrate the potential efficiency of our method sP-PRSM (1.7) by solving the 
Zi-regularized least square problem: 

min \\\Ax-b\\l +p\\x\\x, p > 0 , ( 6 . 1 ) 

where A £ K mxn i s the data matrix, m is the number of the data points, n is the number of features, 
i £ r is the vector of feature coefficients to be estimated and ||x||i := \ x i\i b £ ^ the 

observation vector and p £ R is the regularization parameter. Given n, m = n/2 and a p-sparse vector 
x £ ffi. m (p is the number of nonzero elements in x over m), the Matlab codes for generating the data of 
( 6 . 1 ) are given as 

xbar = sprandn(n,1 ,p) ; D = randn(m,n); 

D = D*spdiags (1./sqrt (sum(D. ~2) ) ’ ,0 ,n,n) ; °/„ normalize columns 
b = D*xbar + sqrt(0.001)*randn(m,1); 
mu = 0.1* norm(D’*b, ’inf’). 

By introducing an auxiliary variable y £ M", we reformulate the problem (6.1) as 

™ Bn \\\ Ax - b\\l + v\\y\\i, s.t. x-y = 0 . ( 6 . 2 ) 

a;Gl n , / 

We consider to apply the sP-PRSM (1.7) to solve (6.2). In our implementation, we always choose T = 0 
and /3 = 1. Starting from x° = y° = 0 and A 0 = 0, with some suitably chosen proximal matrix S, the 
iterative scheme is given as 

' x k+1 = (A T A + /3I + S)-\A T b + t3y k + X k + Sx k ), 

< A fc+ ^ = A fc - a(3(x k+1 - y k ), 

V k+1 = S^/p{x k+1 - A fe+ i//3), 
s A fc+1 = X k+ i - j/3(x k+1 - y k+1 ), 

where the shrinkage operator is defined as S u {y)i = sgn(pi) • max{|z/j| — j/, 0 } and sgn(pj) = 1 if yt > 0 
and sgn(p i ) = —1 if y, < 0. According to (4.4), we stop the iterative scheme when 

max{||x fc+1 - y k+1 1|, || y k+1 - y k \\} < 10 “ 6 (6.4) 

or the iterative counter k > 1000 . 
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Considering that the role of a and 7 in the sP-PRSM (1.7) are equal, we can easily extend the 
convergence domain to 


a € 


0 . 


l + y/5 


, 7 G 0, 


1 — a + i/(l + a) 2 + 4(1 — a 2 ) 


(6.5) 


Note that [13] also establish the convergence of the sP-PRSM (1.7) with 5 = 0 and T = 0 when 

a € (—1, 0 ), 7 £ ( 0 , a 2 — a — 1 ), or a £ ( 0 , 1 ), 7 £ (a 2 — a — 1 , 0 ). ( 6 - 6 ) 


The union domain defined by (6.5) and ( 6 . 6 ) will be considered below. We generate the mesh grid with 
respect to a and 7 by equally dividing the intervals [—1,0] and [0,1.618] into 10 parts, respectively. We 
set n = 2000, p = 0.2. The corresponding scatter diagram is depicted in Figure 1. For consideration 
of space, only the detailed results for (6.5) are shown in Table 1. From this table, we see that when 
a + 7 = 1.618 and min (a, 7) > 0.485, the sP-PRSM (1.7) always perform best; when a and 7 satisfy 
( 6 . 6 ), the results are not so good. 



Figure 1: n = 2000, p = 0.2, the intervals [—1,0] and [0,1.618] are both equally divided into 10 parts. 
Note that the ‘iter’-axis and ‘alpha’-axis are both in reverse order. 


Based on the above observation, we below focus on the choice of a = 0.618 and 7 = 1 . For simplicity, 
we call the corresponding sP-PRSM (1.7) as sP-PRSM*. Recall that the classical ADMM always take 
the dual stepsize as 1 or 1.618 (named by ADMM-1, ADMM-2), covered by our sP-PRSM by taking 
a = 0 , 7 =lora = 0,7 = 1.618, we next compare these three methods for solving ( 6 . 1 ). 

We first consider the sP-PRSM (1.7) without proximal terms, namely, 5 = 0. The comparison results 
are shown in Table 2. Comparing with ADMM-1, sP-PRSM* can always reduce the number of iterations 
in 40%. While as compared to ADMM-2, sP-PRSM* can reduce the number of iterations at about 40% 
for the hard case when the sparsity level p is 0.2 or 0.3, and lead to about 2-7% reduction for the easy 
case when p is 0 . 1 . 

We now consider the sP-PRSM (1.7) with semi-positive proximal matrix 5, which takes the form as 
S = p({S- 1)1 - ^A t a) with i = 1 . 01 A max (J + ^A t A). (6.7) 
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Table 1: The number of iterations taken by sP-PRSM (1.7) with different a and 7 for solving (6.1) with 
n = 2000, p = 0.2, means that the corresponding a and 7 do not satisfy (1.8). 


n. a 

7 \\ 

0.000 

0.162 

0.324 

0.485 

0.647 

0.809 

0.971 

1.133 

1.294 

1.456 

1.618 

0.000 

- 

384 

189 

124 

92 

76 

66 

61 

56 

62 

65 

0.162 

384 

189 

124 

92 

73 

63 

56 

52 

48 

51 

- 

0.324 

189 

124 

92 

73 

61 

55 

49 

45 

43 

60 

- 

0.485 

124 

92 

73 

61 

53 

48 

44 

40 

60 

- 

- 

0.647 

92 

73 

61 

53 

48 

44 

40 

61 

246 

- 

- 

0.809 

75 

63 

54 

48 

44 

40 

61 

246 

- 

- 

- 

0.971 

64 

55 

48 

43 

40 

60 

246 

- 

- 

- 

- 

1.133 

60 

52 

45 

40 

60 

243 

- 

- 

- 

- 

- 

1.294 

56 

48 

43 

61 

242 

- 

- 

- 

- 

- 

- 

1.456 

60 

50 

62 

- 

- 

- 

- 

- 

- 

- 

- 

1.618 

63 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 


Table 2: Comparison among the number of iterations taken by ADMM-1, ADMM-2 and sP-PRSM* for 
solving (6.1) with S = 0 


n 

P 

ADMM-1 

ADMM-2 

sP-PRSM* 

ratiol(%) 

ratio2(%) 

2000 

0.1 

65 

43 

40 

61.5 

93.0 

2000 

0.2 

63 

63 

39 

61.9 

61.9 

2000 

0.3 

85 

93 

53 

62.4 

57.0 

4000 

0.1 

68 

43 

42 

61.8 

97.7 

4000 

0.2 

67 

62 

42 

62.7 

67.7 

4000 

0.3 

80 

88 

49 

61.3 

55.7 

8000 

0.1 

70 

44 

43 

61.4 

97.7 

8000 

0.2 

66 

62 

41 

62.1 

66.1 

8000 

0.3 

69 

79 

42 

60.9 

53.2 
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Now we name ADMM-1 and ADMM-2 as sPADMM-1 and sPADMM-2, respectively. The results are 
shown in Table 3. Note that in this case, sPADMM-1 performs better than sPADMM-2, while sP-PRSM* 
is slightly better than sPADMM-1. 

Table 3: Comparison among the number of iterations taken by sPADMM-1, sPADMM-2 and sP-PRSM 
for solving (6.1) with semi-positive S 


n 

V 

sPADMM-1 

sPADMM-2 

sP-PRSM 

ratiol(%) 

ratio2(%) 

2000 

0.1 

236 

245 

231 

97.9 

94.3 

2000 

0.2 

401 

408 

385 

96.0 

94.4 

2000 

0.3 

574 

579 

549 

95.6 

94.8 

4000 

0.1 

195 

201 

189 

96.9 

94.0 

4000 

0.2 

395 

402 

380 

96.2 

94.5 

4000 

0.3 

542 

548 

518 

95.6 

94.5 

8000 

0.1 

210 

220 

206 

98.1 

93.6 

8000 

0.2 

392 

399 

377 

96.2 

94.5 

8000 

0.3 

491 

498 

471 

95.9 

94.6 


Finally, we follow the way in [15] to generate an indefinite proximal matrix S as 

S = P(Z- 1)/ - A t A, with £ = A max (/ + i A t A ). (6.8) 

Now we name ADMM-1 and ADMM-2 as iPADMM-1 and iPADMM-2, respectively. We list the results 
in Tabic 4. From this table, we can see that sP-PRSM* performs best among the three methods. On 
average, it can lead to 5% reduction in the number of iterations as compared to iPADMM-1 and 10% 
reduction as compared to iPADMM-2. We also remark that for our test instances, no matter choosing 
the semi-definite or indefinite S, ADMM-1 is always better than ADMM-2. Besides, for each method, the 
version with indefinite S in (6.8) can always bring out 40-50% improvement in the number of iterations 
as compared to the version with semi-definite S in (6.7). This coincides with the observation in [15]. 

Table 4: Comparison among the number of iterations taken by iPADMM-1, iPADMM-2 and sP-PRSM 
for solving (6.1) with indefinite S 


n 

P 

iPADMM-1 

iPADMM-2 

sP-PRSM 

ratiol(%) 

ratio2(%) 

2000 

0.1 

124 

138 

122 

98.4 

88.4 

2000 

0.2 

231 

240 

216 

93.5 

90.0 

2000 

0.3 

342 

348 

315 

92.1 

90.5 

4000 

0.1 

105 

116 

101 

96.2 

87.1 

4000 

0.2 

228 

237 

213 

93.4 

89.9 

4000 

0.3 

322 

328 

297 

92.2 

90.5 

8000 

0.1 

107 

122 

106 

99.1 

86.9 

8000 

0.2 

226 

235 

211 

93.4 

89.8 

8000 

0.3 

288 

296 

267 

92.7 

90.2 


From Table 4 we can observe that we can use indefinite matrices in the subproblems (1.7a) and 
(1.7c), which can further improve the efficiency of the algorithm. In fact, under certain conditions on the 
problems’ data and suitable requirements on the matrices, we can theoretically establish its convergence. 
Since the analysis is very similar to those in this paper, we do not describe them here. 
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7 Conclusions 


In this paper, we proposed a modification of the Peaceman-Rachford splitting method by introducing 
two different parameters in updating the dual variable, and by introducing semi-proximal terms to the 
subproblems in updating the primal variables. We established the relationship between the two parame¬ 
ters under which we proved the global convergence of the algorithm. We also analyzed the sublinear rate 
convergence under ergodic and non-ergodic senses, respectively. Under further conditions of one of the 
objective functions, we proved the linear convergence of the algorithm. Finally, we reported extensive 
numerical results, indicating the efficiency of the proposed algorithm. 

Note that the parameters a and 7 are essential to the efficiency of the algorithm, which should be 
variable along with the iteration. Allowing the parameter a and 7 varying with the process of the iterate 
may give us the freedom of choosing them in a self-adaptive manner. Suitable updating rules are among 
our future research tasks. 

On the other hand, considering that the main task in the algorithm is to solve the ^-optimization 
problem (e.g., (1.5a)) or the ^-optimization problem (e.g., (1.5c)), solving them in an inexact manner 
may improve the efficiency of the algorithm. An approximate version of the proposed sP-PRSM with 
practical accuracy criteria is also our future research topic. 
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